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(54) MESSAGE EXCHANGE PROCESSING DEVICE 

(57)Abstract: I g I 

PURPOSE: To enable the sound of a received message to synthesize interactively at a receiving L—J 
side by specifying kinds of sound synthesized and outputted at the receiving side every arbitrary 
character string in character information of a transmitting message. 

CONSTITUTION: When a transmitting message in which control information including an identifier 
indicating kinds of sound or kinds of effective sound is written in arbitrary points of character 
information is sent to its own device of this side, after the message is received by a communication 
control section 8, it is stored in a message processing buffer 103, also a sound elements data group and 
an effective sound group which are attached to the stored message are registered in a sound elements 
data base 106 and an effective sound data base 107 by a message processing section 10. Then, a sound 
synthesizing section 6 fetches successively control information from character information of the 
received message, and sound synthesizing of the character string or synthesizing of effective sound is 
performed by making use of the sound elements data group in the sound elements data base 106 or the 
effective sound data group in the effective sound data base 107 indicated by the identifier in the control 
information. 



* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1]. It has the following and is characterized by making it direct to synthesize voice from a 
character string of a corresponding section in the message using a voice element group peculiar to a 
voice kind which said identifier contained in said control information in the message shows to a device 
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of a transmission destination of said message. A message-switching device which outputs as a sound 
text in a message which exchanged messages and received among other devices via a channel by 
combination of a voice element. 

A message processing means to create a message for transmission to which control information for 
specifying an output with a sound which the identifier shows to an arbitrary part in text which is the 
target of transmission including an identifier which shows an audio kind was set. 
A transmitting means which transmits said message created by this message processing means. 

[Claim 2]A message-switching device which outputs as a sound text in a message which exchanged 
messages and received among other devices via a channel by combination of a voice element, 
comprising: 

A reception means which receives a message of addressing to a self-device transmitted from a device 
besides the above. 

In an arbitrary part in text of said message received by this reception means. A voice synthesis means 
which synthesizes voice from an applicable character string in the message using a voice element group 
peculiar to a voice kind which said identifier contained in this control information shows when control 
information for specifying an output with a sound which the identifier shows including an identifier 
which shows an audio kind is set up. 

[Claim 3]A message-switching device which outputs as a sound text in a message which exchanged 
messages and received among other devices via a channel by combination of a voice element, 
comprising: 

A message processing means to create a message for transmission to which control information for 

specifying an output with a sound which the identifier shows to an arbitrary part in text which is the 

target of transmission including an identifier which shows an audio kind was set. 

A transmitting means which transmits said message created by this message processing means. 

A reception means which receives a message of addressing to a self-device transmitted from a device 

besides the above. 

A voice synthesis means which synthesizes voice from an applicable character string in the message 
using a voice element group peculiar to a voice kind which said identifier contained in this control 
information shows when said control information is set up into said message received by this reception 
means. 

[Claim 4]The message-switching device according to claim 1 or 3, wherein a voice element group 
peculiar to a voice kind which said identifier contained in said control information in the message shows 
is added to said message transmitted by said transmitting means. 

[Claim 5]The message-switching device according to claim 2 or 3, wherein a voice element group 
peculiar to a voice kind which said identifier contained in said control information in the message shows 
is added to said message received by said reception means. 

[Claim 6]It has further a voice element group registration means for registering a voice element group 
for every voice kind used for voice synthesis by said voice synthesis means, The message-switching 
device according to claim 5 characterized by registering a same sound voice element group into said 
voice element group registration means when said voice element group is added to said message 
received by said reception means. 

[Claim 7]In a message-switching device which outputs as a sound text in a message which exchanged 
messages and received among other devices via a channel by combination of a voice element, An 
identifier which shows a kind of a sound or sound effect to an arbitrary part in text which is the target of 
transmission is included, A message processing means to create a message for transmission to which 
control information for specifying an output by a sound or a sound effect which the identifier shows was 
set, A transmitting means which transmits said message created by this message processing means is 
provided, It synthesizes voice from a character string of a corresponding section in the message using a 
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voice element group peculiar to a voice kind which said identifier contained in said control information 
in the message shows to a device of a transmission destination of said message, Or a message-switching 
device characterized by making it direct to compound a sound effect using sound effect data peculiar to 
a sound effect kind which said identifier shows. 

[Claim 8]A message-switching device which outputs as a sound text in a message which exchanged 
messages and received among other devices via a channel by combination of a voice element, 
comprising: 

A reception means which receives a message of addressing to a self-device transmitted from a device 
besides the above. 

In an arbitrary part in text of said message received by this reception means. When control information 
for specifying an output with a sound which the identifier shows including an identifier which shows an 
audio kind is set up, It synthesizes voice from an applicable character string in the message using a voice 
element group peculiar to a voice kind which said identifier contained in this control information shows, 
A voice synthesis means which compounds a sound effect using sound effect data peculiar to a sound 
effect kind which said identifier contained in this control information shows when control information 
for specifying an output by a sound effect which the identifier shows including an identifier which 
shows a kind of sound effect is set up. 

[Claim 9]A message-switching device which outputs as a sound text in a message which exchanged 
messages and received among other devices via a channel by combination of a voice element, 
comprising: 

A message processing means to create a message for transmission to which control information for 
specifying an output by a sound or a sound effect which the identifier shows to an arbitrary part in text 
which is the target of transmission including an identifier which shows a kind of a sound or sound effect 
was set. 

A transmitting means which transmits said message created by this message processing means. 

A reception means which receives a message of addressing to a self-device transmitted from a device 

besides the above. 

When said control information which specifies voice response in said message received by this reception 
means is set up, It synthesizes voice from an applicable character string in the message using a voice 
element group peculiar to a voice kind which said identifier contained in this control information shows, 
A voice synthesis means which compounds a sound effect using sound effect data peculiar to a sound 
effect kind which said identifier contained in this control information shows when said control 
information which specifies a sound effect output is set up. 

[Claim 10]In said message transmitted by said transmitting means. The message-switching device 
according to claim 7 or 9, wherein sound effect data peculiar to a voice element group peculiar to a 
voice kind which said identifier contained in said control information in the message shows, or a sound 
effect kind is added. 

[Claim ll]In said message received by said reception means. The message-switching device according 
to claim 8 or 9, wherein sound effect data peculiar to a voice element group peculiar to a voice kind 
which said identifier contained in said control information in the message shows, or a sound effect kind 
is added. 

[Claim 12] A voice element group registration means for registering a voice element group for every 
voice kind used for voice synthesis by said voice synthesis means, It has further a sound effect 
registration means for registering sound effect data for every sound effect kind used for voice synthesis 
by said voice synthesis means, When said voice element group is added to said message received by 
said reception means, The message-switching device according to claim 1 1 characterized by registering 
the sound effect data into said sound effect registration means when a same sound voice element group 
is registered into said voice element group registration means and said sound effect data is added. 
[Claim 13]The message-switching device according to any one of claims 1 to 12, wherein said control 
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information comprises text containing a predetermined mark or a sign which shows that sympathy news 
is control information. 



[Translation done.] 
* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application]This invention relates to the message-switching device which can exchange a 
character, a sound, a picture message, etc. using channels, such as a Local Area Network or a telephone 
line. 
[0002] 

[Description of the Prior Art]In recent years, it is briskly exchanged increasingly using channels, such as 
a Local Area Network (LAN) or a telephone line, in the message of the various kinds included the 
graphic SUIME 1 JI information created with the text created with the document preparation device etc., 
an image edit device, etc. 

[0003]Besides a means to exchange a message for the device which realizes message switching between 
another message-switching devices which separated the channel, The means which can simplify the 
procedure which transmits a message by a sending person enabling it to choose in the case of message 
transmission, seeing a list of a transmission destination, The means which can transmit by one operation 
if it is the same contents even when transmitting a message to two or more partners, Various facilities 
which are needed in the case of message switching are given by the means which displays an item for 
the message which received on arrival order, the means which classifies and stores the message which 
received and can be referred to at any time, etc. 

[0004]The means which furthermore records with a microphone etc. the sound and various sounds 
which human being emits these days, and carries out digital conversion of this, A means to carry out 
analogue conversion of the information recorded by digital conversion processing on the contrary, to 
carry out [ sound / a sound or ], and to reproduce by a loudspeaker etc., The voice element of the 
ultrashort time amount which analyzed and collected human being's voice elements, and the 
corresponding utterance sign are stored beforehand, By the device by the method which combines the 
voice element corresponding to an utterance sign after changing into an utterance sign from text, and 
makes text utter as a sound also appearing, and combining this and a message-switching device. 
Message switching which combined a character, a sound, a picture, etc. can be easily performed now 
using channels, such as LAN and a telephone line. 

[0005]Unlike the method of communication by a telephone, even when this kind of message-switching 
device has an absent partner, it has the feature that a message is automatically accumulated in a receiver 
and can read the accumulated message at any time with directions of a receiver. 
[0006]Although the message-switching device by the method which has beforehand an utterance sign 
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corresponding to the voice element registered beforehand, combines the voice element corresponding to 
an utterance sign after changing into an utterance sign from text, and makes text utter as a sound has also 
appeared, There was no method besides synthesizing voice in this message-switching device using the 
voice element data with which the receiver was provided beforehand. 
[0007J 

[Problem(s) to be Solved by the Invention]Thus, in the conventional message-switching device, since it 
only synthesized voice using the voice element data in which the receiver was beforehand provided with 
the text in the message sent from the transmitting side, there was a problem that the output in the sound 
which the sending person meant could not be performed. 

[0008] When a scenario, a conference note, etc. were conventionally sent as a message, it was impossible 
to have compounded with a sound interactively by a receiver (that is, perform voice synthesis as if two 
or more human beings were having a dialog). It was also impossible to have outputted the sound effect 
etc. which cannot be expressed with the text in a message by a receiver. 

[0009]This invention was made in consideration of the above-mentioned situation, and the purpose, In 
[ can specify the kind of sound which should be carried out a synthetic output by a receiver for every 
arbitrary character strings in the text used as a transmission object, have, and ] a receiver, An applicable 
character string can be outputted with the sound which the sending person meant, and it is in providing 
the message-switching device which can synthesize voice from dialogue sentences, such as a scenario 
and ******, interactively for each character string of every especially. 

[0010]Even if other purposes of this invention do not have the voice element group beforehand by a 
receiver by adding a voice element group peculiar to the appointed voice kind to an outgoing message, 
there are in providing the message-switching device which can perform voice synthesis by the appointed 
voice kind. 

[001 l]When the purpose of further others of this invention is to save the voice element group added and 
sent to the outgoing message and the output in the same kind of sound is specified by another message 
after that, Even if the voice element group is not added, it is in providing the message-switching device 
which can perform voice synthesis by the appointed voice kind. 

[0012]The purpose of further others of this invention is to provide the message-switching device which 
can reproduce the sound effect etc. which can be specified in the text which serves as a transmission 
object about a sound effect besides a sound, and cannot be expressed with text by a receiver. The 
purpose of further others of this invention has specification of a sound and a sound effect in providing 
the message-switching device which can be easily performed with the feeling which inputs a character 
by a sending person. 
[0013] 

[Means for Solving the Problem and its Function]A message-switching device of this invention to 
achieve the above objects in an arbitrary part in text which is the target of transmission. A message 
processing means to create a message for transmission to which control information for specifying an 
output with a sound (or sound effect) which the identifier shows including an identifier which shows an 
audio kind (or kind of sound effect) was set, A transmitting means which transmits a created message 
for transmission, and a reception means which receives a message of addressing to a self-device 
transmitted from other devices, When the above-mentioned control information is set up into this 
incoming message, according to this control information, It had a voice synthesis means (or a sound 
effect is compounded using sound effect data peculiar to a kind of sound effect) which synthesizes voice 
from an applicable character string in the message using a voice element group peculiar to a voice kind 
which an identifier contained in this control information shows. 

[0014]In such composition, a sending person will fill in beforehand control information which specifies 
voice synthesis by a voice kind corresponding to the words for every words, if text which should be 
transmitted is a scenario. Control information which specifies a sound effect output in a kind of sound 
effect which a sending person meant as a corresponding section in text which needs a sound effect is 
filled in beforehand. Such control information comprises a character string which shows a 
predetermined mark or a sign, and voice kinds (for example, Mr. A's sound, Mr. B's sound, etc.) or 
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sound effect kinds (for example, a rain sound, a sound of a wind, etc.) which shows that sympathy news 
is control information, for example. For this reason, it can enter briefly in text using an editor etc. 
[0015]Text as which control information was written down in each part as mentioned above is processed 
into a message for prescribed format transmission by a message processing means, and is transmitted by 
transmitting means. 

[0016]If this transmitted message is received by reception means of a message-switching device of an 
address, a voice synthesis means in the device will perform voice synthesis according to that control 
information as follows, extracting sequential-control information from text in an incoming message. 
That is, a voice synthesis means will synthesize voice from a character string of a corresponding section 
in an incoming message using a voice element group peculiar to the voice kind, if an identifier in the 
control information shows a voice kind whenever it extracts control information out of text. For 
example, if Mr. A's sound is specified, a voice element required for voice synthesis of the character 
string will be chosen and combined out of a voice element group which analyzed Mr. A's voice element 
and were collected, and, thereby, the character string will be outputted with Mr. A's sound. If an 
identifier in extracted control information shows a kind of sound effect, a voice synthesis means will 
compound a sound effect using sound effect data peculiar to the sound effect kind. For example, if a 
sound effect of rain of Type 1 is specified, the synthetic output of the sound effect of rain of the 
corresponding type 1 will be carried out using data of the sound effect. 

[0017]Thus, a sending person only writes down control information containing an identifier in arbitrary 
parts in text made into a transmission object, and can specify voice synthesis by a voice kind or a sound 
effect which the identifier shows. 

[0018]In a receiver, whenever control information is extracted from text in an incoming message, it is 
switched to an output of applicable character strings (words etc.) with a sound which a sending person 
meant, or an output of a sound effect which a sending person meant. For this reason, dialogue sentences, 
such as a scenario and a conference note, can be reproduced now by two or more interactive sounds and 
sound effects, and it becomes possible to help contents understanding of an incoming message by an 
addressee dramatically. When a voice element group or sound effect data specified by control 
information is not registered into a receiver, the above-mentioned reappearance becomes impossible. 
[0019]Then, this invention creates an outgoing message to which a voice element group specified by 
control information set up into text and sound effect data were added, and transmits to an address. When 
an output in a sound of the same kind next or a sound effect is specified by saving a voice element group 
and sound effect data which were transmitted by doing in this way by a receiver, even if I do not have a 
corresponding voice element group or sound effect data sent, an output in an appointed sound or a sound 
effect is attained. 
[0020] 

[ExampleJDrawing 1 is a block diagram showing the composition of the message-switching device 
concerning one example of this invention. 

[0021]In drawing 1, 1 is indicators, such as a CRT display and a liquid crystal display, and 2 is character 
input parts, such as a keyboard. The input of various commands and control information besides various 
characters is presented with the character input part 2. 3 is voice input parts, such as a microphone, and 4 
is voice output parts, such as a loudspeaker. The voice processing part which performs analog-to-digital 
conversion processing of speech information in which 5 was inputted by the voice input part 3, 6 is a 
speech synthesis section which compounds a sound or a sound effect using the voice element group or 
sound effect data constellation registered into the voice element database 106 or the sound effect 
database 107 mentioned later, and is outputted to the voice output part 4. 

[0022J7 is message store parts, such as an external storage for storing message data. The list (message 
storing situation list) 71 in which the storing situation of the incoming message within the storage 7 is 
shown is placed by the message store part 7. 

[0023]It is a message processing section by which communication control parts, such as a LAN 
controller, a modem, etc. with which 8 manages the communications control between this device and 
other message-switching devices, and 9 control channels, such as a LAN cable and a telephone line, and 
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10 controls this whole device. 

[0024]The transmission buffer 81 for carrying out the temporary storage of the send data and the receive 
buffer 82 for carrying out the temporary storage of the received data are formed in the communication 
control part 8. When the transmission buffer 81 and the receive buffer 82 communicate via other 
message-switching devices and channels 9, they are used by the message processing section 10. 
[0025]The voice data buffer 101 for storing temporarily in the message processing section 10 the speech 
information exchanged between the voice processing parts 5, The character data buffer 102 for storing 
temporarily the text exchanged between the indicator 1 and the character input part 2, The message 
processing buffer 103 for storing temporarily the message data exchanged between the message store 
part 7 and the communication control part 8, The originator information buffer 104 for storing originator 
information, such as a number (device number) of a self-message-switching device, an addresser, and 
time of origin, is formed. The destination database 105 with which the number (receiving set number) of 
the message-switching device of an address and the message transmission point and the conversion table 
with an addressee name (destination name) were registered into the message processing section 10 
again, The voice element database 106 with which much voice element data is stored, and the sound 
effect database 107 with which much sound effect data is stored are formed. Next, operation of the 
composition of drawing 1 is explained with reference to drawing 2 thru/or drawing 8. Message 
transmission operation when the message-switching device shown in introduction and drawing 1 
becomes the transmitting side is explained with reference to the flow chart of drawing 2. 
[0026]First, if a sending person tells the text message in which the control information which specifies 
an audio change etc. was written down, and also a concrete target, The control information which directs 
to synthesize voice from text by the voice element group (for example, voice element group which 
analyzed Mr. A's voice element and were collected) of the appointed voice kind, Or the text message in 
which the control information which is outputted by the appointed sound effect etc., and which carries 
out thing directions was written down is created, for example using a document preparation device etc. 
This control information consists of text set up between predetermined mark (sign) "<" which shows 
that sympathy news is control information, ">", and this "<" and ">" in this example. This text is 
positioned as an identifier (identifier) which shows an audio kind (for example, it is shown that it is Mr. 
A's sound if text is "Mr. A"), or the kind of sound effect. 

[0027]Drawing 3 (a) and (b) shows an example of the text message which did in this way and was 
created. In the text message of drawing 3 (a), compounding the following character string "it's fine 
today, isn't it", for example by the mark of "<A Mr. >" using the voice element data constellation which 
analyzed the voice element of "Mr. A" and were collected is directed. In the text message of drawing 3 
(b), using the data of the sound effect of "1 of ************ by the mark of "1> of < *********** i s 
directed, for example. 

[0028]A sending person can fill in such control information arbitrarily into the usual message using an 
editor etc. Change is also possible. While a sending person has a dialog with the display screen of the 
indicator 1 of this device, this entry is performed and operating the character input part 2 can also make 
a change. When reproducing dialogue sentences, such as a scenario and a conference note, by a receiver 
by this, it enables a sending person to specify arbitrary speech information in a text message. 
[0029]It is considered as the thing of explanation in which control information is written down per line 
for convenience. In this method, if the character string which is the target of composition with the same 
kind of sound straddles two lines, the same control information is attached for every line. The entry 
position of this control information shall be just before an applicable character string. 
[0030]The message processing section 10 inputs the text message which the sending person created 
from the character input part 2 or a floppy disk drive unit (not shown) (Step SI), and stores the message 
in the message store part 7 (Step S2). 

[0031]Next, the message processing section 10 goes into transfer preparation, and makes the 
transmission instructions of a message inputted previously, and directions of an address perform by 
operation of the character input part 2 by a sending person (Step S3). 

[0032]The message processing section 10 receives the directions (from a sending person) inputted from 
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the character input part 2, The inside of the transmission object text message stored in the message store 
part 7 is searched, The voice element data constellation of all the kinds specified by the control 
information in the message (for example, if the "<Mr. A> "mark and "<Mr. B>" mark shown in drawing 
3 (a) is in a text message) All the voice element data registered by "Mr. A" and all the voice element 
data registered by "Mr. B" are obtained from the voice element database 106 (step S4). 
[0033] When the sound effect is also directed within the message, the message processing section 10 
obtains the complex data (for example, - text ~ a message - inside - drawing 3 - (-- b --) - being 
shown — " — < — ********** — one — > — " — a mark — it is -- if — the data of the sound effect of "1 of 
**********") of the directed sound effect from the sound effect database 107 in the above-mentioned 
step S4. 

[0034]The message processing section 10 changes this obtained data into prescribed format voice 
information data, combines with a previous text message, and creates the message (transmitting message 
data) of the form which actually transmits (Step S5). The temporary storage of the message data of this 
transmission form is carried out to the message processing buffer 103. 

[0035]Then, if the message processing section 10 performs the check of address information, the check 
of the contents by the composite tone of the message which transmits, and the last check of message 
transmission by a dialog with the user using indicator 1 grade and is satisfactory, The data transfer of the 
message processing buffer 103 is directed to the communication control part 8. When checking the 
contents by the composite tone of a message, voice synthesis by the speech synthesis section 6 is 
performed based on the voice information data in a message, and the composite tone is outputted to it 
from the voice output part 4. 

[0036]The communication control part 8 transmits the message data stored in the message processing 
Duffer 103 to the transmission buffer 81 one by one with the transmission instructions from the message 
processing section 10, The data is transmitted to the message-switching device of a transmission 
destination via the channel 9 from the buffer 81 (Step S6). 

[0037]Drawing 4 shows the form of the message data transmitted by this message-switching device. As 
shown in a figure, message data (outgoing message), The number of bytes of the data. The shown total 
number of bytes 401, the receiving set number 402 which shows the device number of the message- 
switching (that is, it becomes receiver) device of a transmission destination, the addressee name 403, the 
sender number 404 which shows the device number of the message-switching (that is, it becomes 
origination side) device of a transmitting agency, the addresser name 405, And it has each information 
on the time of origin 406. Message data further, It has each information on 409 voice data which show 
the number of 407 text data bytes, the character information 408, and voice data in which the number of 
bytes of a character information portion is shown, 410 voice data bytes which show the number of bytes 
of a voice data (voice information data) portion, and the voice information data 411. 
[0038]The voice information data 411 consists of a number of voice data which 409 voice data shows. 
Each voice data serves as the number (voice number) 412 assigned to the data in order, the identifier 413 
peculiar to the data, and the number of bytes 414 which shows the number of bytes of a actual voice data 
portion (actual voice data) from the actual voice data 415. The identifier 413 corresponds to "1 of 
**********" under "<Mr. A> "" <1 of **********> G f "Mr. A", and drawing 3 (b)" mark of drawing 3 
(a). [ under mark ] The actual voice data 415 consists of a voice element data constellation or a sound 
effect data constellation. Next, message reception operation when the message-switching device shown 
in drawing 1 becomes a receiver is explained with reference to the flow chart of drawing 5. 
[0039]First, the communication control part 8 is supervising the channel 9, if the message of the form 
shown in drawing 4 via the channel 9 is transmitted, receives the message and once stores it in the 
receive buffer 82 (Step SI 1). 

[0040]Next, the communication control part 8 judges whether the message stored in the receive buffer 
82 is addressing to a self-device based on the receiving set number 402 (refer to drawing 4) in the 
message which received (Step S12), and if it is addressing to a self-device, it will notify that to the 
message processing section 10. 

[0041]Thereby, the message processing section 10 transmits the data of the receive buffer 82 to the 
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message processing buffer 103 (Step SI 3). At this time, the message processing section 10 continues 
(Step S14) sending the data (message) stored in the receive buffer 82 to the message processing buffer 
103 until it reaches the data volume shown with the total number of bytes 401 (refer to drawing 4) in an 
incoming message. 

[0042]The contents of this message processing buffer 103 are transmitted to the message store part 7, 
and the message processing section 10 stores them, if it finishes transmitting all incoming messages to 
the message processing buffer 103 (Step S15). The message processing section 10 registers the storing 
situation in the message store part 7 of this message into the message storing situation list 71 in Step S 15 
simultaneously. That is, the message processing section 10 matches the addressee name (destination 
name) of an applicable message, the message number which shows the message of what position to the 
address it is, and the storing position in the message store part 7 of an applicable message, respectively, 
and registers them into the message storing situation list 71. 

[0043]The example of contents of this message storing situation list 71 is shown in drawing 6. In the 
example of this drawing 6, two messages to a destination name "hiro" are stored, and it is shown in the 
field to which the 1st message (a message number is 1) begins from the 100th street in the message store 
part 7 that the 2nd message is stored to the field which similarly begins from the 1002nd street. It is 
shown that one message to "koya" is stored and the message is stored in the field which begins from the 
3003rd street in the message store part 7. 

[0044]Next, it is received as mentioned above and the operation in the case of performing a voice 
synthesis output (message reproduction to depend) is explained with reference to the flow chart of 
drawing 7 and drawing 8 based on the character information 408 and the voice information data 411 
(refer to drawing 4) in the message stored in the message store part 7. 

[0045]The message processing section 10 displays the list of messages stored in the message store part 7 
on the indicator 1 based on the storing situation in the message storing situation list 71, and makes an 
addressee choose a message to refer to first (Step S21, S22). Thereby, an addressee becomes possible 
[ operating the character input part 2 and carrying out selected designation of the message to refer to ] 
from the list of messages displayed on the indicator 1. Here, if the indicating input of the purport that it 
does not choose by the character input part 2 occurs, the message processing section 10 will end 
processing. 

[0046]On the other hand, when there is an input of the selection instructing of arbitrary messages by the 
character input part 2, the message processing section 10 displays the inquiry screen of whether to need 
voice synthesis on the indicator 1, and makes an addressee choose whether it synthesizes voice. 
[0047]Here, if the indicating input of the purport that it does not synthesize voice by the character input 
part 2 occurs, the message processing section 10 will only display the contents of the character 
information 408 (refer to drawing 4) in the message by which selected designation was carried out on 
the indicator 1, and will end processing (Step S23). On the other hand, the following operations are 
performed when the indicating input of the purport that it synthesizes voice by the character input part 2 
occurs. 

[0048]First, the message processing section 10 transmits the message by which selected designation was 
carried out from the message store part 7 to the message processing buffer 103, and stores it in the buffer 
103 (Step S24). Next, the message processing section 10 refers to 409 voice data in the message stored 
in the message processing buffer 103, the voice data in which the voice data which has a voice element 
data constellation among the voice data for the data number (from -- the becoming voice information 
data 41 1) has a group of sound effect data in the voice element database 106 is registered into the sound 
effect database 107 one by one (Step S25). After this step S25 is completed, control moves from the 
message processing section 10 to the speech synthesis section 6. 

[0049]Then, the speech synthesis section 6 refers to 410 text data bytes in the message stored in the 
message processing buffer 103 (Step S26), A read pointer is set as the place of the beginning of the 
character information 408 in the message (Step S27), and voice synthesis is started so that it may state 
below one by one from the position. 

[0050]The speech synthesis section 6 is first taken out out of the character information 408 as an object 
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character string of composition of the character string for one line which the present pointer points out 
(Step S28). Next, the speech synthesis section 6 looks for control information (pair of shown mark "<" 
and ">") out of the taken-out character string (Step S29). 

[0051] Supposing mark "<" made into the purpose and ">" exist, the speech synthesis section 6, The 
voice element database 106 or the sound effect database 107 is searched as the identifier 413 which 
shows drawing 4 the text set up between the "<"and ">", The voice element (it is equivalent to actual 
voice data 415 of drawing 4) data constellation or sound effect data constellation which should be used 
is determined (Step S30). 

[0052]Next, the portion except the control information of the character strings taken out when the 
speech synthesis section 6 was a case where a voice element data constellation was used (in this 
example.) After performing processing of the common knowledge which analyzes the syntax of the 
character string which follows control information, and is changed into an utterance symbol string, 
Voice synthesis which chooses the voice element data corresponding to each of that utterance sign from 
the voice element data constellations determined at Step S30 (inside of the voice element database 106), 
and is combined is performed (Step S31). On the other hand, if it is a case where a sound effect data 
constellation is used, it will synthesize voice using the sound effect data constellation determined at Step 
S30(StepS31). 

[0053]On the other hand, when control information ("<", ">" which are shown) does not exist in the 
character string taken out at Step S28, After performing processing of the common knowledge which the 
speech synthesis section 6 analyzes the syntax of the character string, and is changed into an utterance 
symbol string, Voice synthesis which chooses the voice element data corresponding to each of that 
utterance sign from the voice element data constellations of the standard beforehand registered into the 
voice element database 106, and is combined is performed (Step S31). This synthesized speech is 
changed into an analog signal, and is outputted by the voice output part 4. 

[0054]If the speech synthesis section 6 investigates whether there is any following alphabetic data (Step 
S32) and is still in the target character information 408 when Step S31 is performed, it will return to 
Step S28 and will perform the voice synthesis of one line as follows. The speech synthesis section 6 
repeats voice synthesis to the termination of the character information 408 similarly hereafter. After a 
series of processings by the speech synthesis section 6 are completed, control moves to the message 
processing section 10, and it returns to the processing for message selection (Step S21). 
[0055] When the outgoing message which uses as the character information 408 the text message shown, 
for example in drawing 3 (a) by the above is sent, In a receiver, it synthesizes voice using the voice 
element data constellation which the next character string "it's fine today, isn't it" of the "<A Mr. >" 
mark filled in as control information analyzed Mr. A's voice element, and were collected in the message. 
Similarly, it synthesizes voice from the next character string "it is said that yes, but it rains tomorrow" of 
a "<B Mr. >" mark using the voice element data constellation which analyzed Mr. B's voice element and 
were collected. 

[0056] When the outgoing message which uses as the character information 408 the text message shown 
in drawing 3 (b) is sent, In the receiver, it synthesizes voice from the character string "Mr. A said then." 
in which control information is not written down probably (addition) using the voice element group 
currently standardly prepared in the receiver, and the next character string "typhoon of the "<A Mr. >" 
mark is approaching, outside rains. " -- it synthesizes voice using the voice element data constellation 
which analyzed Mr. A's voice element and were collected. Next, about "l>of <**********" mark, a 
corresponding sound effect is compounded using a sound effect data constellation peculiar to 1 of 
********** 

[0057]Thus, since the information on two or more sorts of sounds and sound (sound effect) is 
compoundable according to the control information which comprises text, it is very effective when 
helping an understanding of a message content to a message reception person. 
[0058]Although said example explained as what adds the voice element data constellation or sound 
effect data constellation of the sound or sound effect specified by the control information filled in into 
the character information 408 required for composition to an outgoing message as the voice information 
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data 411, and transmits, It does not restrict to this. For example, the voice element data constellation or 
sound effect data constellation which transmitted once, Since it registers with the voice element database 
106 or the sound effect database 107 of the receiver, in subsequent message transmission, it is not 
necessary to necessarily send already about transmitted a voice element data constellation or a sound 
effect data constellation. 

[0059]By asking the kind (identifier) of the voice element data constellation which the device has, or 
sound effect data constellation to other message-switching devices, It is also possible to judge whether 
according to this list, it is necessary to create the list of kinds of the voice element data constellation 
which each device has, or sound effect data constellation, and to send the voice element data 
constellation or sound effect data constellation specified by control information. 
[0060JIn said example, when the voice kind was specified by the identifier in control information, 
explained as what adds all the voice element data constellations peculiar to the voice kind to an outgoing 
message, and sends them, but. You may make it send only voice element data required to synthesize 
voice from an applicable character string. 

[0061] Although said example explained the case where an audio kind etc. were switched according to 
the information specified by a sending person, An addressee also becomes possible [ specifying the 
voice kind at time of reproducing a message, etc. ] by changing arbitrarily the control information 
written down in the character information 408 in an incoming message by an editor etc. 
[0062] 

[Effect of the InventionJAs explained in full detail above, in this invention, it has composition which 
transmits the message for transmission to which the control information for specifying an output with 
the sound which the identifier shows to the arbitrary part in the text which is the target of transmission 
including the identifier which shows an audio kind was set. 

Therefore, a sending person only writes down control information in the arbitrary part in the text which 
should be transmitted, and can specify composition with the sound of the kind corresponding to the part 
which self meant for every character string to a receiver. 

A sending person can also specify outputs, such as a sound effect which cannot be expressed with text, 
by applying the control information for specifying the output by a sound effect including the identifier 
which shows the kind of sound effect. 

[0063] According to this invention, in a receiver, the output in the sound effect which could compound 
the applicable character string with the sound of the kind which the sending person meant according to 
the control information set as the arbitrary part of the text in an incoming message, and the sending 
person meant can also be performed. And since control information is set up for every arbitrary 
character strings and voice synthesis by arbitrary voice kinds can be specified, in a receiver. Dialogue 
sentences, such as a scenario and a conference note, can be interactively compounded with the sound of 
specification suitable for each for each character string of every, and an understanding of a message 
content can be effectively helped to a message reception person. 

[0064] In this invention, a voice element group (or sound effect data peculiar to a sound effect kind) 
peculiar to the voice kind specified by the control information filled in into the text which is the target of 
transmission is considered as the composition added to an outgoing message. 

Therefore, in a receiver, even if it does not have beforehand the voice element group (or sound effect 
data), voice synthesis (or composition of a sound effect) by the appointed voice kind can be performed 
using the voice element group (or sound effect data). 

[0065] In this invention, the voice element group (or sound effect data) added and sent to the outgoing 
message is considered as the composition saved by a receiver so that reuse is possible. 
Therefore, when the output in a sound (or sound effect) of the same kind next is specified, even if I do 
not have a corresponding voice element group (or sound effect data) sent, composition with the 
appointed sound (or sound effect) can be performed. 

[0066]Text constitutes control information from this invention. 
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Therefore, audio (or sound effect) specification can carry out easily with the feeling which inputs a 
character by a sending person. 

Since an addressee can also change control information arbitrarily to the text in an incoming message, 
composition with the sound (or sound effect) suitable for an addressee's liking can also be performed. 



[Translation done.] 
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1. This document has been translated by computer. So the translation may not reflect the original 
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2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing l]The block diagram showing the composition of the message-switching device concerning 
one example of this invention. 

[Drawing 2]The flow chart for explaining the procedure of the message transmission processing in the 
example. 

[Drawing 3]The figure showing the example of a text message in which control information was filled 
in. 

[Drawing 4]The figure showing the structure of the message data transmitted and received with the 
device of drawing 1. 

[Drawing 5]The flow chart for explaining the procedure of the message reception processing in the 
example. 

[Drawing 6]The figure showing the example of contents of the message storing situation list 71 placed 
by the message store part 7 in drawing 1. 

[Drawing 7]The figure showing a part of flow chart for explaining the procedure of the voice synthesis 
processing (message reproduction processing) in the example. 
[Drawing 8]The figure showing the remainder of the flow chart. 
[Description of Notations] 

1 [ — A speech synthesis section, II — Message store part, ] ~ An indicator, 2 — A character input part, 
4 -- A voice output part, 6 8 [ -- A message storing situation list, 103 / -- A message processing buffer, 
106 / ~ A voice element database, 107 / -- Sound effect database. ] -- A communication control part, 9 - 
A channel, 10 — A message processing section, 71 
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1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DRAWINGS 



[Drawing 




[Drawing 2] 
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[Drawing 6] 
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[Drawing 3] 



[Drawing 5] 
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[Drawing 4] 
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[Drawing 7] 
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[Drawing 8] 
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